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Amendments to the Claims: 

Following is a complete listing of the claims pending in the application, as amended, 
which replaces all prior versions and listings of claims in the application: 

1 . (Currently Amended) One or more computer-readable media storing a computer 
program that, when executed by one or more processors on a mobile computer, causes the one or 
more processors to: 

display to a user operating the mobile computer a subset of a plurality of steps in an order 
to be performed by a-the user; 

alter an appearance of a current step in the subset of steps that needs to be performed by 
the user to distinguish the current step from other steps in the subset; 

receive information about a curr e nt cont e xt of the user from a cont e xt awar e n e ss 
compon e nt that r e c e iv e s includes sensed information from multiple sources such that sensed 
information from differing sources is at least partially in conflict; and that 

automatically build a model of the current context of the user by mediat ing es amongst the 
sensed information from the multiple sources to build a mod e l of th e curr e nt cont e xt of th e us e r ; 

in response to the rec e ived information about built model of the current context of the 
user, alter one or more of the subset of steps that needs to be performed by the user by altering 
multiple interaction elements that affect interactions with the user for the current step, the 
interaction elements including: 

instructions associated with handling the current step; 
presentation of instructions associated with handling the current step; 
available choices which are used to handle the current step; 
methods that a user may use to handle the current step; and 
means by which the user provides input to complete the current step; 
allow the user to input data corresponding to the current step; 

when input data is not received from the user for the current step and information 
received from the context awareness component indicates that the user is currently distracted, 
further altering one or more of the interaction elements for the current step in such a manner as to 
be less cognitively burdensome for the user; 
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scroll, in response to user input of data corresponding to the current step, the plurality of 
steps so that a new subset of the plurality of steps is presented to the user; and 

amend the step for which the data input was received from the user with indications of 
that data input. 

2. (Original) One or more computer-readable media as recited in claim 1, wherein 
the computer program further causes the one or more processors to: 

alter, in response to user input of data corresponding to the current step, the appearance 
of another step as necessary to identify the new current step in the subset of steps that needs to be 
performed by the user. 

3. (Original) One or more computer-readable media as recited in claim 1, wherein 
altering the appearance of the current step comprises marking the current location with a ball. 

4. (Original) One or more computer-readable media as recited in claim 1, wherein 
altering the appearance of the current step comprises displaying the current step differently than 
other steps in the subset. 

5. (Original) One or more computer-readable media as recited in claim 1, wherein 
altering the appearance of the current step comprises replacing the current step with a set of one 
or more input options for the current step. 

6. (Original) One or more computer-readable media as recited in claim 1, wherein 
altering the appearance of the current step comprises superimposing, on the current step, a set of 
one or more input options for the current step. 

7. (Original) One or more computer-readable media as recited in claim 1, wherein 
the computer program further causes the one or more processors to: 

replace, in the subset, the display of the current step with a display of the input data. 
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8. (Original) One or more computer-readable media as recited in claim 1, wherein 
the computer program further causes the one or more processors to: 

display a current processing marker that identifies which step in the subset of steps is 
currently being processed by the one or more processors. 

9. (Original) One or more computer-readable media as recited in claim 1, wherein 
the one or more computer-readable media comprise a computer memory of a wearable computer. 

10. (Currently Amended) A computer-implemented method for displaying 
information to a user in a manner based on a modeled current context of the user, comprising: 

automatically generating a modeled current context of a user operating a computing 
system based on information about the user obtained from multiple sources, the modeled current 
context of the user including one or more indications of a predicted mental state of the user; 

displaying to the user a list of items to be handled by a-the user in a particular order and ? 

identifying one item in the list of items that is as a current item; 

receiving information sensed from an environment of the user by one or more sources; 
automatically about a updating the modeled current context of the user based at least in 
part on the received sensed information ; 

in response to the r e c e iv e d information ab o u t t h e updated modeled current context of the 
user, automatically determining one or more of multiple elements to alter regarding interactions 
with the user and altering the determined elements, the multiple elements including: 
instructions associated with handling the current item; 
presentation of instructions associated with handling the current item; 
available choices which are used to handle the current item; 
methods that a user may use to handle the current item; and 
means by which the user provides input to complete the current item; 
receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 
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11. (Original) A method as recited in claim 10, wherein displaying the list of items 
comprises displaying at least one item corresponding to a task that has already been performed 
and at least one item corresponding to a task that still needs to be performed by the user. 

12. (Original) A method as recited in claim 10, wherein displaying the list of items 
comprises displaying, after the user input is received, the user input in place of the corresponding 
item. 

13. (Original) A method as recited in claim 10, wherein displaying the list of items 
comprises displaying only a subset of the list of items at any given time. 

14. (Original) A method as recited in claim 13, further comprising scrolling through 
the list of items to display different subsets as items in the list are handled by the user, 

15. (Original) A method as recited in claim 10, further comprising displaying a 
current processing marker identifying an item in the list of items corresponding to a current user 
input being processed. 

16. (Original) A method as recited in claim 10, wherein the list of items comprises a 
list of tasks to be completed by the user, and wherein handling of an item by the user comprises 
the user completing the task. 

17. (Original) A method as recited in claim 16, wherein the list of tasks comprises a 
list of prompts corresponding to data to be entered into the computer by the user. 

18. (Original) A method as recited in claim 10, wherein the list of items comprises a 
list of prompts of words to be spoken by the user, and wherein handling of an item by the user 
comprises speaking one or more words corresponding to the prompt. 
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19. (Currently Amended) One or more computer-readable memories containing a 
computer program that is executable by a processor to perform a method comprising: 

displaying to a user operating a computing system a list of items to be handled by a-the 
user in a particular order; 

identifying one item in the list of items that is a current item; 

receiving information about the user from one or more sources external to the user and 
automatically updating a modeled current context of the use r based at least in part on the 
received information ; 

in response to the r e c e iv e d information about th e updated modeled current context of the 
user, altering multiple of a plurality of interaction elements for the current item based on the 
current context, the interaction elements including: 

instructions associated with handling an item; 
presentation of instructions associated with handling an item; 
available choices which are used to handle an item; 
methods that a user may use to handle an item of the list; and 
mechanisms by which the user provides input to complete an item; 
receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 

20-34. (Canceled.) 

35. (Currently Amended) A system comprising: 
one or more output devices; 

a user interface component, coupled to an output device, causing a user interface to be 
output on the output device; 

a module that automatically models a current context of a user operating the system based 
at least in part on information related to the user that is received from one or more sources other 
than the user; 
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a module that provides information about the modeled current context of the user to the 
user interface component; 

wherein the user interface includes a list portion in which a list of a plurality of items to 
be handled by a user are output; 

wherein the user interface further includes a current location marker identifying one of 
the items in the list as the current item that needs to be handled by the user; 

wherein the user interface component further automatically updates the current location 
marker to identify a new item in the list in response to the user handling the current item in the 
list; 

wherein the user interface component, in response to the received information about the 
current context of the user, alters one or more of the following elements: 
instructions associated with handling an item; 
presentation of instructions associated with handling the item; 
available choices which are used to handle the item; 
methods that a user may use to handle an item of the list; and 
means by which the user provides input to complete the item; and 
wherein the determination of which elements to alter is in response to the received 
information about the current context of the user. 

36. (Original) A system as recited in claim 35, wherein the user interface component 
further replaces, after the user has handled the current item, a user input in place of the current 
item. 

37. (Original) A system as recited in claim 35, wherein the user interface includes 
only a subset of the list of the plurality of items at any given time. 

38. (Original) A system as recited in claim 37, wherein the user interface component 
further scrolls through the list of items to display different subsets as items in the list are handled 
by the user. 
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39. (Original) A system as recited in claim 35, wherein the user interface component 
further displays, as part of the user interface, a current processing marker identifying an item in 
the list that is currently being processed by the system. 

40. (Original) A system as recited in claim 35, wherein the list of a plurality of items 
comprises a list of a plurality of tasks to be completed by the user, and wherein handling of an 
item by the user comprises the user completing the task. 

41. (Original) A system as recited in claim 40, wherein the list of tasks comprises a 
list of prompts corresponding to data to be entered into the system by the user. 

42. (Original) A system as recited in claim 40, wherein the user interface component 
is implemented in software. 

43-59. (Canceled.) 

60. (Previously Presented) The method of claim 10 wherein the identifying comprises 
superimposing, on the display of the current item in the list, a set of one or more input options 
corresponding to the item. 

61. (Currently Amended) The method of claim 60 wherein the receiving of the user 
input comprises receiving, as the input corresponding to the current item, one of the input 
options from the set of one or more input options. 

62. (Previously Presented) The method of claim 10 wherein the particular order is 
altered in response to the receiving of the information about the current context of the user. 

63. (Previously Presented) The method of claim 10 wherein the determining of which 
of the multiple elements to alter is based at least in part on information about the current context 
of the user. 
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64. (Previously Presented) The method of claim 63 wherein the current context of the 
user includes an estimate of cognitive availability of the user derived at least in part from user 
response time. 

65. (Previously Presented) The method of claim 10 including, after receiving current 
context information for the user regarding a current ability of the user to handle items, further 
altering the multiple elements to reduce a cognitive burden of handling items. 

66. (Previously Presented) The method of claim 65 wherein the further altering is not 
performed if the user acts promptly. 

67. (Currently Amended) The method of claim 10 wherein the determining of which 
of the multiple elements to alter is performed based on the current context so as place minimal 
cognitive burden on the user. 

68. (Previously Presented) The method of claim 10 wherein information about the 
item for which the user input was received is amended with indications of the user input. 

69. (Previously Presented) The method of claim 10 wherein the displaying and 
presentation may be performed using any appropriate means of communicating with the user 
including audio and/or video. 

70. (Previously Presented) The method of claim 10 wherein further alteration is done 
to amend a modality of user input after receiving current context information for the user 
regarding an ability of the user to handle items using different modalities. 

71. (Previously Presented) The method of claim 10 wherein further alteration is done 
to amend a modality of display or presentation after receiving current context information for the 
user regarding an ability of the user to handle items using different modalities. 
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72. (Previously Presented) The method of claim 10 wherein the determining of which 
of the multiple elements to alter is performed so as to allow the user to recognize presented user 
input that will handle the item. 

73. (Currently Amended) The method of claim 10 wherein the curr e nt cont e xt of th e 
us e r is det e rmin e d hy a r e mot e modul e based on s e ns e d information from one or more sources 
include multiple sources that sense information about the user and/or about an environment of 
the user. 

74. (Previously Presented) A method comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to receiving information about the current context of the user, 
available choices which are used to handle the item; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 

75. (Previously Presented) A method comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to receiving information about the current context of the user, 
presentation of instructions associated with handling the item; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 
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76. (Previously Presented) A method comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to receiving information about the current context of the user, 
methods which a user may use to handle an item of the list; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 

77. (Previously Presented) A method comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to receiving information about the current context of the user, 
mechanisms by which the user provides input to complete the item; 
receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 

78. (Previously Presented) A method comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to receiving information about the current context of the user, 
instructions to the user associated with handling items; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 
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79. (Previously Presented) A computer-readable medium whose contents cause a 
computing device to perform a method, comprising: 

displaying a list of items to be handled by a user in a particular order; 
identifying one item in the list of items that is a current item; 
receiving information about a current context of the user; 

altering, in response to the receiving information about the current context of the user, 
one or more of the following elements: 

instructions associated with handling an item; 

presentation of instructions associated with handling the item; 

available choices which are used to handle the item; 

methods which a user may use to handle an item of the list; and 

mechanisms by which the user provides input to complete the item; 

wherein the determination of which elements to alter is in response to the received 
information about the current context of the user; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 



80. (Previously Presented) A computing device with output devices, comprising: 
a first module configured to display a list of items to be handled by a user in a particular 
order and to identify one item in the list of items that is the current item; 

a second module configured to receive information about the current context of the user 
and to alter, in response to the receiving information about the current context of the user, one or 
more of the following elements: 

instructions associated with handling an item; 
the presentation of instructions associated with handling the item; 
the available choices which are used to handle the item; 
the methods which a user may use to handle an item of the list; or 
the means by which the user provides input to complete the item; 
wherein the determination of which elements to alter is in response to the received 
information about the current context of the user; and 



38 



Application No. 09/879,829 

Reply to Office Action dated September 12, 2005 

a third module configured to receive a user input corresponding to the current item, and 
to update, in response to receiving the user input, the identification of the one item that is the 
current item to indicate the next item in the list of items as the current item. 

81. (Previously Presented) The computing device of claim 80 wherein the computing 
device is a wearable computer. 

82. (New) A method comprising: 

displaying to a user operating a computer a list of items to be handled by the user in a 
particular order, the displayed list including one item in the list that is visually identified as a 
current item; 

receiving information about a modeled current context of the user, the modeled current 
context of the user including information about a predicted mental state of the user and a 
predicted activity of the user; 

altering, in response to receiving information about the modeled current context of the 
user, available choices which are available to the user to handle the current item; 

receiving a user input corresponding to the current item; and 

updating, in response to receiving the user input, the identification of the one item that is 
the current item to indicate the next item in the list of items as the current item. 

83. (New) The method of claim 82 wherein the predicted mental state of the user 
indicates that the user is currently distracted. 
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